Image processing

ABSTRACT

A method and apparatus for the processing of video data in which the appearance characteristics of picture elements are modified using digital apparatus such as a color processor which provides a plurality of channels having respective processing means, each channel being adapted normally to operate to process a discrete range of characteristics, wherein the method comprises the step of operating the apparatus in an alternative mode in which at least two of the channels are used to carry out identical processing steps, spatial segments of the video data being split between said at least two channels to thereby increase the speed of processing.

FIELD OF INVENTION

The present application relates to image processing and in particular toelectronic and computer techniques to enhance the editorial effects thatcan be produced with motion picture images resulting from film or video.

BACKGROUND OF THE INVENTION

Machines to produce electrical signals from motion picture film havebeen known for many years. References exist showing that such processeswere known in the 1920's and used by John Logie Baird. Examples ofcurrent machines (‘telecine machines’) to perform this function includethe URSA ‘Diamond’ machine manufactured by Cintel International Ltd. ofWare, Hertfordshire, and the ‘SPIRIT Datacine’ from Philips, Darmstadt,Germany.

These systems permit some editorial adjustment to the electrical imagesproduced from the film to be made. These functions usually include theadjustment of gain (white level), lift (black level), and gamma(contrast and midtone). Some telecine machines also contain primary andsecondary colour correction systems. Primary colour correction is theadjustment of the primary colour content of the image. For example, allareas containing red may be made redder i.e. the level of red in thewhole image may be increased. This will necessarily affect the greytones which are composed of equal components of red, green, and blue.Secondary colour correction is the adjustment of particular colours inthe image and, for example, distinctions may be made between areascontaining solely red, and areas containing red in conjunction withother colours, such as grey areas.

It is commonly preferred for telecine users and owners to purchaseseparate colour correction systems to those included in some telecinemachines. An example of such a system is the ‘DCP’ system manufacturedby Pandora International Ltd. Many aspects of this system are disclosedin U.S. Pat. No. 5,450,500, International patent application WO95/12289, and others.

As is disclosed in U.S. Pat. No. 5,450,500, the DCP system is capable ofidentifying a particular area of an image and, if required, adjustingthe appearance of the image to achieve a desired effect. The area to beadjusted may be identified by one or more criteria including theluminance, hue, saturation, position or texture of one or more of itsconstituent pixels. The DCP is equipped with eight channels each ofwhich can be configured to perform a separate adjustment on anindividually defined area of the image. Each pixel in the stream ofdigital video data from a telecine machine or video source is testedsequentially by each successive channel of the DCP for correspondencewith the predetermined criteria stored in that channel. If the pixelmeets the criteria of a particular channel it is tagged for adjustmentin accordance with the adjustment parameters defined in that channel.The adjustment parameters may include adjustment of one or more of theluminance, hue, saturation or position of the pixel. Thus, each of thechannels will adjust the appearance of the pixel under test if it meetsthe adjustment criteria of that channel. The channels are designed suchthat each channel in turn can test the input video data stream at asufficiently high speed that the DCP produces a corrected video datastream at broadcast rate.

There is, however, a trend in the television and video post-productionindustry to move away from the traditional ‘clocked’ processing oftelevision images, whereby video frame rates are matched to thebroadcast rate of 50 or 60 Hz, so that each television line is processedin exactly one 625th or 525th of the picture frame rate. Modern trendsare to process images asynchronously, where data calculations take aslong as necessary, provided that the average data processing ratematches the expected rates. Sometimes, it is not even necessary to matchaverage real time or broadcast rates, as some processing in this‘computer’ format is acceptable in non-real time.

Traditionally, video data has been stored on video tape and, inparticular, digital video data has been stored on digital video tape.However, digital video tape can only be used to record data at real timerates. The advent of large capacity magnetic storage media such a harddisks has meant that video data can be stored and accessed at ratesfaster than real time. Video colour correctors on the other hand remainconfigured to process video data at real time rates.

SUMMARY OF THE INVENTION

Viewed from one aspect the present invention provides a method for theprocessing of video data in which the appearance characteristics ofpicture elements are modified using digital apparatus which provides aplurality of channels having respective processing means, each channelbeing adapted normally to operate to process a discrete range ofcharacteristics, wherein the method comprises the step of operating theapparatus in an alternative mode in which at least two of the channelsare used to carry out identical processing steps, spatial segments ofthe video data being split between said at least two channels to therebyincrease the speed of processing.

Preferably the digital apparatus is a digital colour processing unit inwhich video data may be passed in sequence through selection means foreach channel which determines whether pixels are to be modified inaccordance with selection criteria for that channel, and wherein thedigital colour processing unit is further provided with routing meansfor operating the unit in an alternative mode in which data is passed inparallel through at least two of the selection means, said at least twoof the selection means having identical selection criteria and theassociated channels modifying selected pixels in an identical way.

Viewed from another aspect the present invention provides a method ofprocessing a stream of video data, wherein the data stream is split intoa plurality of sequential portions and the portions are routed to acorresponding plurality of processing units each configured to performan identical operation on the data.

Viewed from another aspect the present invention provides apparatus forthe processing of a continuous stream of video data, the apparatuscomprising a plurality of processing units for processing the video dataand routing means for routing successive portions of the data streamrespectively to at least two of the processing units, when said at leasttwo processing units are configured to perform an identical processingoperation on the video data.

In accordance with the present invention the video data stream may beprocessed faster than the individual capability of each processing unitas each processing unit will be operating on only a portion of the videodata. The processing units will therefore operate on the video data inparallel when they are performing identical operations.

The number of portions into which the data stream is split will bedetermined in practice by reference to the number of availableprocessing units and the number of separate simultaneous operationsrequired. For example, in a system of six processing units, or channels,required to carry out only two modifications, the video data stream willbe split into three successive portions and each portion will be routedthrough two processing units, each of the two processing unitsperforming a respective modification.

Reconstituting means may also be provided for reconstructing the splitportions of the data stream back into a single stream after processing.

The number of processing channels used may be altered dynamically. Forexample, an image may require only a simple correction for one part ofthe image, such as the adjustment of one colour, and this may be doneusing one processor only. Elsewhere in the image there may be a regionwhere a complicated change is required, involving lots of separatecolour adjustments. Additional processing channels may be called in andat least two channels used simultaneously to carry out the same steps.Subsequently these processors may be released.

Adopting computer processing techniques can lead to great productivityincreases if intelligent routing is applied to the mapping of algorithmsonto the available hardware in a system such as the Pandora DCP. Take asan example the processing of data of a film resolution image in ahardware system such as the DCP. With film resolution work there is noconcept of real time, as there is in broadcast television, because theprocessing operations required are so computationally intensive. It ismerely required to process images as cost effectively and efficiently aspossible. As disclosed in U.S. Pat. No. 5,450,500, colour correctionsystems often have six or more dedicated channels for colour correction.Traditionally, it has been common to have each of these channelsconfigured to process one sixth of the colour space or colour huecircle. The colour hue circle can be thought of as a continuous circularfield of colours from red to yellow to green to cyan to blue to magentaand back to red. Thus the six conventional channels on a colourcorrector are red, yellow, green, cyan, blue, and magenta. According toU.S. Pat. No. 5,450,500 these channels are, however, completelyconfigurable by the operator. Thus, six different areas of the red hue,for example, can be identified, and processed accordingly.

According to the present invention if the operator wishes to make achange only to one red area, all six of the colour processing enginesmay be configured to process different spatial sixths of the image, toattain processing of the aforementioned film resolution data image inone sixth of the time that would be usual in a conventional synchronouscolour processing system. Thus, the data stream corresponding to theimage may be routed in six sequential portions to respective channels ofthe colour processor. Methods for splitting the image between colourprocessors are many. One possible method is to alternate betweenprocessors along the line direction of the image, feeding the firstpixel on the line to the first processor, the second pixel to the secondprocessor, and so on, such that successive channels receive successivepixels in a pattern that is repeated after the sixth pixel in the line.Another method is to split the lines between processors, such that allof the first line is processed by the first processor, all of the secondline is processed by the second processor, and so on. Alternatives tothese methods are many, and include, for example, the processing of thefirst sixth of the lines of a frame by the first processor, the secondsixth by the second processor, and so on. Alternatively, each line maybe split into six continuous portions, one portion being routed to eachchannel.

Similarly, if the operator wishes to make two alterations, to twoseparate colour regions in the picture, then three times the speed ofprocessing can still be achieved than would have previously beenpossible. In this case, the six channels will be split into two equalgroups each channel in a group performing an identical colour correctionoperation. Each group will receive all of the picture information in thevideo data stream, but each channel within the group will only receiveone third of the pixels in the data stream for processing.

Other areas of processing to which this technique is applicable includethe simple lookup techniques used to alter the gain, lift, and gamma.These functional units can run at many times the speed of the colourchannel processor, which in turn can usually run at real time in a videoconfiguration. Conventionally, in the video domain, there would be nopoint in running adjustments or processing at higher than real timevideo rate, but with the advent of film resolution data, it is useful tobe able to process the data at the maximum rate possible.

Advantageously, some of the channels may be configured in softwarerather than the hardware configuration such as that of the DCP.Particularly advantageously a mixture of software and hardware channelsmay be used with the routing taking into account the relativecapabilities of each channel. For example, to get maximum throughput ofdata in a configuration where four channels of colour processing arerequired, in a hardware/software system that has six channels availableof hardware, it may be possible to run at ‘double speed’ with thehardware, processing colour channel one with hardware channels 1 and 2,colour channel two with hardware channels 3 and 4, colour channel threewith hardware channels 5 and 6, whilst processing the fourth colourchannel in software. There are many other ways in which a mixedenvironment of hardware and software processing elements can be matchedinto a ‘load balanced’ combination, by the use of intelligent routing.

One hardware implementation of the invention could use a plurality of“Pentium”™ or equivalent chips which can operate at speeds of 333 MHz oreven more. Alternatively there could be used specialist processors suchas the TMS 320C6X of Texas Instruments. This range includes both fixedand floating point chips. A third alternative is the “Network Share”range from Analog Devices in the United States. Alternatively, largefield programmable gate array (FPGA) elements could be used, such as the10K150 element from Altera in the United States.

BRIEF DESCRIPTION OF THE DRAWINGS

An embodiment of the invention will now be described with reference tothe accompanying drawings in which:

FIG. 1 is a diagrammatic view of digital colour processing apparatusoperating in a conventional mode; and

FIG. 2 is a diagrammatic view of the apparatus modified and operating ina different mode in accordance with the invention.

DESCRIPTION OF PREFERRED EMBODIMENT

The digital colour processing apparatus is of the general type disclosedin U.S. Pat. No. 5,450,500, the contents of which are incorporatedherein by reference.

As can be seen in FIG. 1 a video signal is fed into processing meanscomprising a series of six channels, only three of which areillustrated, through which the pixels pass sequentially. Each channel isset up with its own selection criteria and its own modificationcriteria. Thus, in channel 1, all red pixels could be selected and thecolour modified. In channel 2 all pixels satisfying a particularcombination of colour properties and spatial coordinates could beselected and modified, and so on for all six channels. The modifiedpixels are then added back to the stream with pixels which have passedthrough without modification.

FIG. 2 shows the system modified in accordance with one embodiment ofthe present invention. In this arrangement a router is employed and thishas disabled the normal sequential path, as denoted by “X”. The routerhas instead been set up to feed the six channels in parallel. In thisparticular embodiment each of the six channels has been set up with thesame selection criteria and the same modifications to be carried out.The router may supply successive pixels to the channels in sequence, orfor example send an entire line of a video image through one channel,the next line through the next channel, and so forth until coming backto the first channel again.

It will be understood by those skilled in the art that the aboveexemplary embodiment does not limit the scope of the present invention.For example, six channels have been described, although of course asystem comprising any number of channels would be improved by theinvention. Furthermore, the invention has been described with particularreference to the Pandora DCP of the type illustrated in U.S. Pat. No.5,450,500 although the invention is equally applicable to any similarimage processing system providing that the routing of the signalsthrough the channels can be configured as required.

What is claimed is:
 1. A method for the processing of a stream of videodata in which the appearance characteristics of picture elements aremodified using digital apparatus which provides a plurality of channelshaving respective processing means, each channel being adapted normallyto operate to process a discrete range of characteristics of the datastream by carrying out a respective different processing step, whereinthe method comprises the steps of: a) splitting the video data streaminto at least two sequential spatial segments; b) providing a first saidsequential spatial segment of said video data stream to a first channel;c) providing a second said sequential spatial segment of said video datastream to a second channel; and d) operating the apparatus in analternative mode in which the first and second channels are used tocarry out identical processing steps on the respective first and secondspatial segments of the video data such that the first and secondspatial segments of the video data are processed in parallel by saidfirst and second channels to thereby increase the speed of processing,wherein the digital apparatus is a digital colour processing unit inwhich the video data may be passed in sequence through respectiveselection means associated with each respective channel, wherein eachselection means determines whether pixels are to be modified inaccordance with selection criteria for that channel, and wherein thedigital colour processing unit is further provided with routing means,the method comprising the further step of: operating the unit in analternative mode in which the first and second sequential spatialsegments of the video data are passed in parallel through at least twoof the selection means, said at least two of the selection means havingidentical selection criteria and the associated channels modifyingselected pixels in an identical way.
 2. Apparatus in the form of adigital colour processing unit for the processing of a continuous streamof video data, the digital colour processing unit comprising: aplurality of processing units for processing the video data; routingmeans for routing successive portions of the data stream respectively toat least two of the processing units, when said at least two processingunits are configured to perform an identical processing operation on thevideo data; selection means associated with each processing unit throughwhich video data may be passed in sequence, which determines whetherpixels are to be modified in accordance with selection criteria for thatprocessing unit; and routing means for operating the digital colourprocessing unit in an alternative mode in which data is passed inparallel through at least two of the selection means, said at least twoof the selection means having identical selection criteria and theassociated processing units modifying selected pixels in the identicalway.